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Calibrating a Measure of Motivation in Learning about Technology 

American society is increasingly technologically oriented. Consequently, our keeping pace 
with other industrialized nations requires educators to insure that students who are future entrants 
into the workforce have the necessary knowledge and skill in learning and using 
technology/computer. However, research and statistical data demonstrated that American 
students have fallen off the pace with regard to mathematics and science achievement. 

Even within American population, it is apparent that there are groups that are not 
performing up to their potential. For example, females (Linn & Hyde, 1988) have been shown to 
achieve less in science and math than white middle class males. Because demographic trends 
suggest that woman and minorities will constitute larger percentages of the workforce in the 
future, it is imperative that we understand why females are not achieving up to their potential and 
are not choosing to pursue careers in techno logy/computer. 

OBJECTIVES 

The present study examines the functioning of an instrument for measuring elementary 
school students’ motivation to learn about technology. If this variable can be measured 
quantitatively, as shown via data fit to a fundamental measurement model (Luce & Tukey, 1964; 
Krantz, Luce, Suppes, & Tversky, 1971; Michell, 1990, 1997; 1999), further research will 
examine potential sources of differences in learning and using technology between female and 
male sixth graders. The theoretical framework of this study is based on Eccles and Wigfield 
(1995) expectancy- value theory, which focuses on cognitive factors in determining achievement 
behaviors. More specifically, differences in self-concept of ability, perception of technology. 
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perception of parental beliefs, and causal attributions (success and failure), are investigated as 
they relate to achievement. 

METHOD 

Participants 

There were 129 (65 males & 64 females) sixth grade students enrolled in an elementary 
school in a southern state. The mean age for the students was 10.4. 

Instrument 

The questionnaire used in this study was revised based on Whang and Hancock (1994) 
and Eccles’ expectancy x value theory. The 36 items in the questionnaire thought to bear on a 
measure of motivation to learn about technology involve self-concept of ability (6 items), 
perception of technology (4 items), intrinsic casual attributions (4 items), extrinsic casual 
attributions (3 items), task-involved motivation (3 items), ego-involved motivation (3 items), 
parent’s perception (7 items), female gender issues (3 items), male gender issues (3 items). These 
items are constructed using a Likert scale anchored by very true (1) and not true at all (4). 
Procedure 

The surveys were group administered orally to students in their homerooms, using 
standardized instructions by a trained graduate student. Participants were told that the purpose of 
the study was to learn what students think using and learning technology/computer is all about 
and how they feel about their ability in using and learning technology/computer. Approximately 
30 min was needed to complete the questionnaire. Care was taken through the surveys to give no 
indication of what would be considered the appropriate or “right” answer. If the student did not 
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understand a question, it was repeated or paraphrased in simpler language, but still no examples 
or suggested answers were given. 

Analysis 

Fundamental measurement theory (Luce & Tukey, 1964; Krantz, Luce, Suppes, & 
Tversky, 1971; Michell, 1990, 1997; 1999) provides a strong program for testing the hypothesis 
that a given variable is quantitative. According to Mundy (Mundy, 1986, p. 392), 

The hallmark of a meaningless proposition is that its truth- value depends on what scale or 
coordinate system is employed, whereas meaningful propositions have truth- value 
independent of the choice of representation, within certain limits. 

Fundamental measurement theory’s approach to testing the quantitative hypothesis examines the 
extent to which mathematical propositions concerning the variable will be meaningful or 
meaningless. In other words, fundamental measurement theory establishes whether a construct’s 
quantitative expression depends upon the particular test or survey questions asked (what might be 
called the “brand” of instrument) and/or upon the particular sample of persons responding to the 
questions. 

A particularly easy to use and convenient way of testing the quantitative hypothesis is via 
fit to a probabilistic conjoint (Rasch) measurement model (Rasch, 1960; Perline, Wright, & 
Wainer, 1979; Wright & Masters, 1982; Andrich, 1988; Fisher & Wright, 1994; Wright, 1999). 
When data fit one of these models, the axioms of simultaneous conjoint measurement and the 
meaningfulness criterion are satisfied. The model employed here is the rating scale model 

nl(p / (1-p)) = b n - dj - kj 





read as the natural logarithm of the odds (p / (1-p)) that difference between the ability b of person 
n is greater than the difficulty d of item i at the level k posed by category j. Survey applications 
typically assume that measures are not affected by any factors other than the properties of the 
questions asked and the attitudes or abilities of the respondents. Fundamental measurement 
models do nothing more than check these assumptions for their tenability in the face of actual 
observations. 

Given the sample size of 129 and the 36-item instrument, overall success in the 
measurement effort will be indicated when the standard deviations of the information-weighted 
and outlier-sensitive model fit statistics are less than 2.0, and when individual scores function as 
sufficient statistics (i.e., the pattern of responses across the items for a person, or the pattern of 
responses across the persons for an item, is reproducible from the score alone) (Smith, 1986, 
1998; Wright & Masters, 1982). 

RESULTS 

Table 1 shows the summary statistics for the student measures and the item calibrations. 
The data matrix of 129 students times the 36 items contains 4,644 possible observations; 98.8% 
of these (4,587) are valid. Raw scores range from 67 to 1 15, within the maximum range of 36 to 
144. The students responded to 35.6 questions on average. 

Overall model fit appears acceptable for the item calibrations, given that the standard 
deviations of the standardized fit statistics are considerably less than 2.0. 

Several students, however, appear to have provided inconsistent responses, as the 
maximum information- weighted (infit) and outlier-sensitive (outfit) statistics are 5.7 and 5.2, 
respectively. Examination of the residuals indicates several highly unexpected responses to items 
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21 and 10, which ask the students about their parents’ opinions concerning girls’ need to study 
technology (21) and concerning whether the respondent hates technology (10). 

Modeled reliability, at .62, allows for barely 2 statistically distinct strata (measurement 
ranges with centers at least three errors apart (Wright & Masters, 1982). The average 
measurement error of .19 is close to that predicted by Rasch generalizability theory for a survey of 
36 four-category items, but the remarkably low measurement standard deviation (SD) of .29 
makes for a measurement separation (ratio of the SD to the error) of only 1 .29. 

Table 2 shows the summary statistics for the rating scale categories. Note that the 
observed counts in each category range from 749 to 1714, and the step calibrations are not 
ordered from less to more as the category labels progress from 1 to 4. 

The distribution of the measures, averaging .18 logits, is illustrated in Figure 1, showing 
that the measures are most closely aligned with the calibrations of the items on the average step of 
the rating scale, which approximates the step from Somewhat True to Not Very True. The 
average measure is about one error above the center of the item scale (0.0). Were the instrument 
better targeted, error would be somewhat lower, and reliability, higher. 

Figure 2 shows the items in measure order, along with the positions of the category 
transitions on the number line. The statements at the bottom of the figure are rated least true, and 
those at the top, most true. Students rate the assertion that their parents do not value technology 
education important for girls untrue, and they deny learning about technology to prevent getting 
in trouble. Students also tend to find statements concerning the technical gender superiority of 



either boys or girls untrue. 
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At the other end of the continuum, statements that the students find very true involve 
learning about technology because it is interesting, obtaining good technology course grades 
because of hard work, expecting good grades in technology classes, and the importance parents 
place on learning about technology. 

Figure 3 shows a box plot of the items’ calibration values by the theoretical construct 
groupings. Items involving task-orientation, ability self-concepts, perceptions of technology, and 
internal causal attributions are rated most true, with statements concerning ego-involved 
motivation, external causal attributions, and gender issues rated least true. 

Analysis of the measure, error, standardize fit statistic, and point biserial correlation 
variances (via ANOVA) revealed few relationships that could clearly be interpreted as statistically 
or substantively significant. The most suggestive findings indicate that the information-weighted 
fit statistics are elevated for black males and Hispanic females (there were no Hispanic male 
respondents and only two Hispanic females), meaning that the survey may not validly measure 
attitudes toward technology for these groups. 

CONCLUSIONS and DISCUSSION 

Based on these data, it seems that our attempt to develop an instrument to measure 
motivation in learning about technology was successful, to certain degree. In general, the survey 
items behaved fairly well as measures of motivation in learning about technology and gender 
differences. The homogeneity of the measures, as indicated by the low SD, and the high model fit 
statistics suggest that further research into the construct and the items bearing on it may be 
warranted. However, eight of the items exhibited low point- biserial correlations. Removing the 
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four worst-fitting cases and another four individual observations on item 21 reduced the 
standardized outfit SD from 2.1 to 1.8, and the standardized infit SD from 2.3 to 2.2. 

Ideally, each category’s probability curve should have its own peak, indicating the point at 
which it is the most probable response for a given range of measures. The curves in Table 2 
indicate that categories 2 and 3 are never the most likely response. Respondents apparently cannot 
distinguish four separate degrees of truth in these statements. Future analyses of these data should 
explore various ways of combining categories to linearize the step calibrations. And if the 
instrument is to be administered to another sample, the response options ought to be expanded 
from the current four to six, with the category labels modified to more clearly demarcate 
increasing amounts. For instance, an Agree/Disagree continuum, with “Veiy Strongly”, 
“Strongly”, and “Mildly” as modifiers, might more clearly convey a wider range of possible 
responses than the current labels’ focus on truth. 
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FIGURE 1 

Technical Motivation Variable Map 
129 students, 36 items, 4 Categories 
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FIGURE 2 

Technical Motivation Expected Responses 
129 students, 36 items, 4 Categories 
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FIGURE 4 

Technical Motivation Principal Components (Standardized Residual) Factor Plot 

129 Persons, 36 Items, 4 Categories 



Factor 1 explains 4.45 of 36 variance units 
Pearson Correlation: -.84 
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TABLE 1 

Technical Motivation Instrument Calibration Summary Statistics 
129 students, 36 items, 4 Categories 
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TABLE 2 

Technical Motivation Summary of Measured Steps on the Rating Scale 
129 students, 36 items, 4 Categories 
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